Lecture 26 - Your project. Deep Learning with H2O. P.2 Build Deep Learning Model for our example
Deep Learning… This lecture is dedicated to the implementation of Deep Learning models into for our Data
Work overview (from previous lecture)
- re-arranging data as matrix
- fitting deep learning models
- testing the models
- saving models
- implementation in our ShinyApp…
The idea will be to use one particular feature e.g. Tubing Process, Impedance to play with. We will not yet implement this in Shiny but rather focus to prepare our data, fit and test the model
Re-arranging data as a matrix
Let’s get our time - series data as a dataframe first…
library(tidyverse)
# ============= READ DATA =================
# Read our small data ...
DF_Data_Recent <- readRDS("DF_Data_Process_Recent.data")
DF_Equipm <- read_csv("DF_EquipmData.csv")
# data frame containing Event Names
DF_EvCode <- read_csv("DF_EvCodeDataProject.csv")
# Data manipulation and saving to the DF_TEMP
DF_TEMP <- DF_Data_Recent %>%
# join to decode equipment serial number
inner_join(DF_Equipm, by = "IDEquipment") %>%
# join to decode Event Code meaning
inner_join(DF_EvCode, by = "EventCode") %>%
# select only column needed
select(StartDate, Name, AnalogVal, EventText)
Then I will plot my data for all 4 machines as just to remember how it looks like…
# creating human readable data and visualize them
DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
ggplot(aes(x = StartDate, y = AnalogVal, col = Name)) + geom_point()+facet_grid(~Name)

Looking on the chart above I can see that machine #1 seems to be the best. I will take that one as a reference to build my Deep Learning Model. I will be extracting this dataset with a filter() function
# extracting only one machine
DF_M1 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #3") %>%
select(StartDate, AnalogVal) %>%
head(50)
Transposing the data
Now we need to transpose our data from long to wide structure! We will ‘forget’ the StartDate values and simply parse the values into matrix of dimension say 150 columns and 20 rows… Of course one need to recover some basic R skills for that :) if not use stackoverflow… (How to turn a vector into a matrix in R?)[https://stackoverflow.com/questions/14614946/how-to-turn-a-vector-into-a-matrix-in-r]
DF_M1 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #1") %>%
select(AnalogVal) %>%
t() %>% # this brings us a matrix
as.vector() %>% # let's make it a vector
head(3000) %>% # only making fixed amount of rows
matrix(nrow = 20, ncol = 150) # transforming that into matrix size 25x200
Wonderful, let’s try to see! our new object as an image!!!
Explore the matrix as a surface!
Let’s use plotly 3D graph to explore what we have got!
library(plotly)
plot_ly(z = DF_M1, type = "surface")
This should be good enough to fit our deep learning model, but before we will do so I will ‘prepare’ my test datasets
A. For machine 2:
DF_M2 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #2") %>%
select(AnalogVal) %>%
t() %>% # this brings us a matrix
as.vector() %>% # let's make it a vector
head(1500) %>% # only making fixed amount of rows
matrix(nrow = 10, ncol = 150) # transforming that into matrix size 25x200
B. For machine 3:
DF_M3 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #3") %>%
select(AnalogVal) %>%
t() %>% # this brings us a matrix
as.vector() %>% # let's make it a vector
head(3000) %>% # only making fixed amount of rows
matrix(nrow = 20, ncol = 150) # transforming that into matrix size 25x200
C. For machine 4:
DF_M4 <- DF_TEMP %>%
filter(EventText == "Tubing Process, resistance Ohm") %>%
filter(Name == "Machine #4") %>%
select(AnalogVal) %>%
t() %>% # this brings us a matrix
as.vector() %>% # let's make it a vector
head(2850) %>% # only making fixed amount of rows
matrix(nrow = 19, ncol = 150) # transforming that into matrix size 25x200
And we can also visualize that to confront:
For Machine 2
plot_ly(z = DF_M2, type = "surface")
For Machine 3
plot_ly(z = DF_M3, type = "surface")
For Machine 4
plot_ly(z = DF_M4, type = "surface")
Fitting the Deep Learning Model
Launch the machine again…
localH2O = h2o.init()
H2O is not running yet, starting it now...
Note: In case of errors look at the following log files:
C:\Users\fxtrams\AppData\Local\Temp\RtmpOi23u1/h2o_fxtrams_started_from_r.out
C:\Users\fxtrams\AppData\Local\Temp\RtmpOi23u1/h2o_fxtrams_started_from_r.err
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
Starting H2O JVM and connecting: . Connection successful!
R is connected to the H2O cluster:
H2O cluster uptime: 2 seconds 479 milliseconds
H2O cluster version: 3.14.0.7
H2O cluster version age: 22 days
H2O cluster name: H2O_started_from_R_fxtrams_utl430
H2O cluster total nodes: 1
H2O cluster total memory: 1.77 GB
H2O cluster total cores: 4
H2O cluster allowed cores: 4
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
H2O Internal Security: FALSE
H2O API Extensions: Algos, AutoML, Core V3, Core V4
R Version: R version 3.2.5 (2016-04-14)
Load datasets to H2O
Then we will download the datasets into h2o. Remember H2O is just operated from R but it’s a computer besides!
# Import train data into the H2O cluster
train_M1 <- as.h2o(x = DF_M1, destination_frame = "train_M1")
# Also import our test datasets for Machines 2 and 3...
test_M2 <- as.h2o(x = DF_M2, destination_frame = "test_M2")
test_M3 <- as.h2o(x = DF_M3, destination_frame = "test_M3")
# Make your own research usign DF_M4!
Building Deep Learning model with autoencoder
Now, once we know how our data looks like we can start to do our Anomaly Model
# Train deep autoencoder learning model on "normal"
# training data, y ignored
normality_model <- h2o.deeplearning(
x = names(train_M1),
training_frame = train_M1,
activation = "Tanh",
autoencoder = TRUE,
hidden = c(50,20,50),
sparse = TRUE,
l1 = 1e-4,
epochs = 100)
|
| | 0%
|
|================================================================================================================| 100%
Calculate MSE from the train dataset
Let’s use this model on our training dataset…

Getting to know what AI see:
# visually see it
test_recon_M1 <- h2o.predict(normality_model, train_M1) %>% as.matrix()
plot_ly(z = test_recon_M1, type = "surface")
Detect Anomaly using MSE value
Now we will try to use test datasets from machine 2
# Compute reconstruction error with the Anomaly
# detection app (MSE between output and input layers)
h2o.anomaly(normality_model, test_M2) %>% as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")
And for machine 3
h2o.anomaly(normality_model, test_M3) %>% as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")
It is now telling us that behaviour of Machine 2 is much different from machine 1. And on Machine 3 we have a clearly peaks that are distinguishable to recover anomaly!
Note: You may try to practice with Machine 4!
Use Anomaly model to reconstruct Test Dataset
Now we can obtain predictions, or physical values using our model. We should provide the model and test dataset
# Note: Testing = Reconstructing the test dataset
test_recon_M3 <- h2o.predict(normality_model, test_M3) %>% as.matrix()
plot_ly(z = test_recon_M3, type = "surface")
we can not definitely see those peaks as clear now however let us be satisfied with the model result and think how to implement our Deep Learning Model in ShinyApp
Saving Deep Learning Model for the future use
To use our model in our ShinyApp we will save it…
if(!file.exists("www/tmp/normality_model.bin")){
h2o.saveModel(normality_model, "www/tmp/normality_model.bin")
h2o.download_pojo(normality_model, "www/tmp", get_jar = TRUE)
}
And let’s not forget to switch off our cluster!
h2o.shutdown(prompt= FALSE)
Conclusion
In this example the Anomaly Detection model was able to output the anomaly in rows 21-23.
It learned on the pattern of many vectors and was able to distinguish the anomaly coming on new dataset
Practical use of this model can be to us function h2o.anomaly. In case the MSE value will be high - the anomaly is detected!
Next step
our next step will be to repeat the procedure but on our machine data.
- re-arranging data as matrix
- fitting deep learning models
- testing the models
- saving models
- implementation in our ShinyApp…
LS0tDQp0aXRsZTogIkxlY3R1cmUgMjYgLSBJbnRvIERlZXAgTGVhcm5pbmcgUDIiDQpvdXRwdXQ6DQogIGh0bWxfZG9jdW1lbnQ6IGRlZmF1bHQNCiAgaHRtbF9ub3RlYm9vazogZGVmYXVsdA0KICBwZGZfZG9jdW1lbnQ6IGRlZmF1bHQNCi0tLQ0KDQpUaGlzIGlzIGFuIFtSIE1hcmtkb3duXShodHRwOi8vcm1hcmtkb3duLnJzdHVkaW8uY29tKSBOb3RlYm9vaw0KDQoNCiMjIyBMZWN0dXJlIDI2IC0gWW91ciBwcm9qZWN0LiBEZWVwIExlYXJuaW5nIHdpdGggSDJPLiBQLjIgQnVpbGQgRGVlcCBMZWFybmluZyBNb2RlbCBmb3Igb3VyIGV4YW1wbGUNCg0KRGVlcCBMZWFybmluZy4uLiBUaGlzIGxlY3R1cmUgaXMgZGVkaWNhdGVkIHRvIHRoZSBpbXBsZW1lbnRhdGlvbiBvZiBEZWVwIExlYXJuaW5nIG1vZGVscyBpbnRvIGZvciBvdXIgRGF0YQ0KDQojIyMjIFdvcmsgb3ZlcnZpZXcgKGZyb20gcHJldmlvdXMgbGVjdHVyZSkNCg0KKiByZS1hcnJhbmdpbmcgZGF0YSBhcyBtYXRyaXgNCiogZml0dGluZyBkZWVwIGxlYXJuaW5nIG1vZGVscw0KKiB0ZXN0aW5nIHRoZSBtb2RlbHMNCiogc2F2aW5nIG1vZGVscw0KKiBpbXBsZW1lbnRhdGlvbiBpbiBvdXIgU2hpbnlBcHAuLi4NCg0KVGhlIGlkZWEgd2lsbCBiZSB0byB1c2Ugb25lIHBhcnRpY3VsYXIgZmVhdHVyZSBlLmcuICoqVHViaW5nIFByb2Nlc3MsIEltcGVkYW5jZSoqIHRvIHBsYXkgd2l0aC4gV2Ugd2lsbCBub3QgeWV0IGltcGxlbWVudCB0aGlzIGluIFNoaW55IGJ1dCByYXRoZXIgZm9jdXMgdG8gcHJlcGFyZSBvdXIgZGF0YSwgZml0IGFuZCB0ZXN0IHRoZSBtb2RlbA0KDQojIyMjIFJlLWFycmFuZ2luZyBkYXRhIGFzIGEgbWF0cml4DQoNCkxldCdzIGdldCBvdXIgdGltZSAtIHNlcmllcyBkYXRhIGFzIGEgZGF0YWZyYW1lIGZpcnN0Li4uDQoNCmBgYHtyLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQ0KbGlicmFyeSh0aWR5dmVyc2UpDQpsaWJyYXJ5KHBsb3RseSkNCg0KIyA9PT09PT09PT09PT09IFJFQUQgREFUQSA9PT09PT09PT09PT09PT09PQ0KIyBSZWFkIG91ciBzbWFsbCBkYXRhIC4uLiANCkRGX0RhdGFfUmVjZW50IDwtIHJlYWRSRFMoIkRGX0RhdGFfUHJvY2Vzc19SZWNlbnQuZGF0YSIpIA0KDQpERl9FcXVpcG0gPC0gcmVhZF9jc3YoIkRGX0VxdWlwbURhdGEuY3N2IikNCiMgZGF0YSBmcmFtZSBjb250YWluaW5nIEV2ZW50IE5hbWVzDQpERl9FdkNvZGUgPC0gcmVhZF9jc3YoIkRGX0V2Q29kZURhdGFQcm9qZWN0LmNzdiIpDQoNCiMgRGF0YSBtYW5pcHVsYXRpb24gYW5kIHNhdmluZyB0byB0aGUgREZfVEVNUA0KREZfVEVNUCA8LSBERl9EYXRhX1JlY2VudCAlPiUgDQogICMgam9pbiB0byBkZWNvZGUgZXF1aXBtZW50IHNlcmlhbCBudW1iZXINCiAgaW5uZXJfam9pbihERl9FcXVpcG0sIGJ5ID0gIklERXF1aXBtZW50IikgJT4lIA0KICAjIGpvaW4gdG8gZGVjb2RlIEV2ZW50IENvZGUgbWVhbmluZw0KICBpbm5lcl9qb2luKERGX0V2Q29kZSwgYnkgPSAiRXZlbnRDb2RlIikgJT4lIA0KICAjIHNlbGVjdCBvbmx5IGNvbHVtbiBuZWVkZWQNCiAgc2VsZWN0KFN0YXJ0RGF0ZSwgTmFtZSwgQW5hbG9nVmFsLCBFdmVudFRleHQpDQpgYGANCg0KVGhlbiBJIHdpbGwgcGxvdCBteSBkYXRhIGZvciBhbGwgNCBtYWNoaW5lcyBhcyBqdXN0IHRvIHJlbWVtYmVyIGhvdyBpdCBsb29rcyBsaWtlLi4uDQoNCmBgYHtyfQ0KIyBjcmVhdGluZyBodW1hbiByZWFkYWJsZSBkYXRhIGFuZCB2aXN1YWxpemUgdGhlbQ0KREZfVEVNUCAlPiUgDQogIGZpbHRlcihFdmVudFRleHQgPT0gIlR1YmluZyBQcm9jZXNzLCByZXNpc3RhbmNlIE9obSIpICU+JSANCiAgZ2dwbG90KGFlcyh4ID0gU3RhcnREYXRlLCB5ID0gQW5hbG9nVmFsLCBjb2wgPSBOYW1lKSkgKyBnZW9tX3BvaW50KCkrZmFjZXRfZ3JpZCh+TmFtZSkNCmBgYA0KDQpMb29raW5nIG9uIHRoZSBjaGFydCBhYm92ZSBJIGNhbiBzZWUgdGhhdCBtYWNoaW5lICMxIHNlZW1zIHRvIGJlIHRoZSBiZXN0LiBJIHdpbGwgdGFrZSB0aGF0IG9uZSBhcyBhIHJlZmVyZW5jZSB0byBidWlsZCBteSBEZWVwIExlYXJuaW5nIE1vZGVsLiBJIHdpbGwgYmUgZXh0cmFjdGluZyB0aGlzIGRhdGFzZXQgd2l0aCBhICoqZmlsdGVyKCkqKiBmdW5jdGlvbg0KDQpgYGB7cn0NCiMgZXh0cmFjdGluZyBvbmx5IG9uZSBtYWNoaW5lDQpERl9NMSA8LSBERl9URU1QICU+JSANCiAgZmlsdGVyKEV2ZW50VGV4dCA9PSAiVHViaW5nIFByb2Nlc3MsIHJlc2lzdGFuY2UgT2htIikgJT4lIA0KICBmaWx0ZXIoTmFtZSA9PSAiTWFjaGluZSAjMyIpICU+JSANCiAgc2VsZWN0KFN0YXJ0RGF0ZSwgQW5hbG9nVmFsKSAlPiUgDQogIGhlYWQoNTApDQpgYGANCg0KIyMjIyBUcmFuc3Bvc2luZyB0aGUgZGF0YQ0KDQpOb3cgd2UgbmVlZCB0byAqKnRyYW5zcG9zZSoqIG91ciBkYXRhIGZyb20gbG9uZyB0byB3aWRlIHN0cnVjdHVyZSEgV2Ugd2lsbCAnZm9yZ2V0JyB0aGUgU3RhcnREYXRlIHZhbHVlcyBhbmQgc2ltcGx5IHBhcnNlIHRoZSB2YWx1ZXMgaW50byBtYXRyaXggb2YgZGltZW5zaW9uIHNheSAxNTAgY29sdW1ucyBhbmQgMjAgcm93cy4uLiBPZiBjb3Vyc2Ugb25lIG5lZWQgdG8gcmVjb3ZlciBzb21lIGJhc2ljIFIgc2tpbGxzIGZvciB0aGF0IDopIGlmIG5vdCB1c2Ugc3RhY2tvdmVyZmxvdy4uLiAoSG93IHRvIHR1cm4gYSB2ZWN0b3IgaW50byBhIG1hdHJpeCBpbiBSPylbaHR0cHM6Ly9zdGFja292ZXJmbG93LmNvbS9xdWVzdGlvbnMvMTQ2MTQ5NDYvaG93LXRvLXR1cm4tYS12ZWN0b3ItaW50by1hLW1hdHJpeC1pbi1yXQ0KDQpgYGB7cn0NCkRGX00xIDwtIERGX1RFTVAgJT4lIA0KICBmaWx0ZXIoRXZlbnRUZXh0ID09ICJUdWJpbmcgUHJvY2VzcywgcmVzaXN0YW5jZSBPaG0iKSAlPiUgDQogIGZpbHRlcihOYW1lID09ICJNYWNoaW5lICMxIikgJT4lIA0KICBzZWxlY3QoQW5hbG9nVmFsKSAlPiUgDQogIHQoKSAlPiUgICMgdGhpcyBicmluZ3MgdXMgYSBtYXRyaXgNCiAgYXMudmVjdG9yKCkgJT4lICMgbGV0J3MgbWFrZSBpdCBhIHZlY3Rvcg0KICBoZWFkKDMwMDApICU+JSAjIG9ubHkgbWFraW5nIGZpeGVkIGFtb3VudCBvZiByb3dzDQogIG1hdHJpeChucm93ID0gMjAsIG5jb2wgPSAxNTApICMgdHJhbnNmb3JtaW5nIHRoYXQgaW50byBtYXRyaXggc2l6ZSAyNXgyMDANCmBgYA0KDQpXb25kZXJmdWwsIGxldCdzIHRyeSB0byBzZWUhIG91ciBuZXcgb2JqZWN0IGFzIGFuIGltYWdlISEhDQoNCiMjIyMgRXhwbG9yZSB0aGUgbWF0cml4IGFzIGEgc3VyZmFjZSENCg0KTGV0J3MgdXNlIGBwbG90bHlgIDNEIGdyYXBoIHRvIGV4cGxvcmUgd2hhdCB3ZSBoYXZlIGdvdCENCg0KYGBge3IsIGV2YWw9RkFMU0UsIGluY2x1ZGU9VFJVRX0NCmxpYnJhcnkocGxvdGx5KQ0KcGxvdF9seSh6ID0gREZfTTEsIHR5cGUgPSAic3VyZmFjZSIpDQoNCmBgYA0KDQohW1ZhbHVlcyBmcm9tIE1hY2hpbmUgMSB1c2VkIGZvciBUcmFpbmluZ11baWQxXQ0KDQoNCg0KVGhpcyBzaG91bGQgYmUgZ29vZCBlbm91Z2ggdG8gZml0IG91ciBkZWVwIGxlYXJuaW5nIG1vZGVsLCBidXQgYmVmb3JlIHdlIHdpbGwgZG8gc28gSSB3aWxsICdwcmVwYXJlJyBteSB0ZXN0IGRhdGFzZXRzDQoNCkEuIEZvciBtYWNoaW5lIDI6DQoNCmBgYHtyfQ0KREZfTTIgPC0gREZfVEVNUCAlPiUgDQogIGZpbHRlcihFdmVudFRleHQgPT0gIlR1YmluZyBQcm9jZXNzLCByZXNpc3RhbmNlIE9obSIpICU+JSANCiAgZmlsdGVyKE5hbWUgPT0gIk1hY2hpbmUgIzIiKSAlPiUgDQogIHNlbGVjdChBbmFsb2dWYWwpICU+JSANCiAgdCgpICU+JSAgIyB0aGlzIGJyaW5ncyB1cyBhIG1hdHJpeA0KICBhcy52ZWN0b3IoKSAlPiUgIyBsZXQncyBtYWtlIGl0IGEgdmVjdG9yDQogIGhlYWQoMTUwMCkgJT4lICMgb25seSBtYWtpbmcgZml4ZWQgYW1vdW50IG9mIHJvd3MNCiAgbWF0cml4KG5yb3cgPSAxMCwgbmNvbCA9IDE1MCkgIyB0cmFuc2Zvcm1pbmcgdGhhdCBpbnRvIG1hdHJpeCBzaXplIDI1eDIwMA0KYGBgDQoNCkIuIEZvciBtYWNoaW5lIDM6DQoNCmBgYHtyfQ0KREZfTTMgPC0gREZfVEVNUCAlPiUgDQogIGZpbHRlcihFdmVudFRleHQgPT0gIlR1YmluZyBQcm9jZXNzLCByZXNpc3RhbmNlIE9obSIpICU+JSANCiAgZmlsdGVyKE5hbWUgPT0gIk1hY2hpbmUgIzMiKSAlPiUgDQogIHNlbGVjdChBbmFsb2dWYWwpICU+JSANCiAgdCgpICU+JSAgIyB0aGlzIGJyaW5ncyB1cyBhIG1hdHJpeA0KICBhcy52ZWN0b3IoKSAlPiUgIyBsZXQncyBtYWtlIGl0IGEgdmVjdG9yDQogIGhlYWQoMzAwMCkgJT4lICMgb25seSBtYWtpbmcgZml4ZWQgYW1vdW50IG9mIHJvd3MNCiAgbWF0cml4KG5yb3cgPSAyMCwgbmNvbCA9IDE1MCkgIyB0cmFuc2Zvcm1pbmcgdGhhdCBpbnRvIG1hdHJpeCBzaXplIDI1eDIwMA0KYGBgDQoNCkMuIEZvciBtYWNoaW5lIDQ6DQoNCmBgYHtyfQ0KREZfTTQgPC0gREZfVEVNUCAlPiUgDQogIGZpbHRlcihFdmVudFRleHQgPT0gIlR1YmluZyBQcm9jZXNzLCByZXNpc3RhbmNlIE9obSIpICU+JSANCiAgZmlsdGVyKE5hbWUgPT0gIk1hY2hpbmUgIzQiKSAlPiUgDQogIHNlbGVjdChBbmFsb2dWYWwpICU+JSANCiAgdCgpICU+JSAgIyB0aGlzIGJyaW5ncyB1cyBhIG1hdHJpeA0KICBhcy52ZWN0b3IoKSAlPiUgIyBsZXQncyBtYWtlIGl0IGEgdmVjdG9yDQogIGhlYWQoMjg1MCkgJT4lICMgb25seSBtYWtpbmcgZml4ZWQgYW1vdW50IG9mIHJvd3MNCiAgbWF0cml4KG5yb3cgPSAxOSwgbmNvbCA9IDE1MCkgIyB0cmFuc2Zvcm1pbmcgdGhhdCBpbnRvIG1hdHJpeCBzaXplIDI1eDIwMA0KYGBgDQoNCkFuZCB3ZSBjYW4gYWxzbyB2aXN1YWxpemUgdGhhdCB0byBjb25mcm9udDoNCg0KRm9yIE1hY2hpbmUgMg0KDQpgYGB7ciwgZXZhbD1GQUxTRSwgaW5jbHVkZT1UUlVFfQ0KcGxvdF9seSh6ID0gREZfTTIsIHR5cGUgPSAic3VyZmFjZSIpDQpgYGANCg0KIVtWYWx1ZXMgZnJvbSBNYWNoaW5lIDIgdXNlZCBmb3IgVGVzdF1baWQyXQ0KDQpGb3IgTWFjaGluZSAzDQoNCmBgYHtyLCBldmFsPUZBTFNFLCBpbmNsdWRlPVRSVUV9DQpwbG90X2x5KHogPSBERl9NMywgdHlwZSA9ICJzdXJmYWNlIikNCmBgYA0KDQohW1ZhbHVlcyBmcm9tIE1hY2hpbmUgMyB1c2VkIGZvciBUZXN0XVtpZDNdDQoNCkZvciBNYWNoaW5lIDQNCg0KYGBge3IsIGV2YWw9RkFMU0UsIGluY2x1ZGU9VFJVRX0NCnBsb3RfbHkoeiA9IERGX000LCB0eXBlID0gInN1cmZhY2UiKQ0KYGBgDQoNCiFbVmFsdWVzIGZyb20gTWFjaGluZSA0IHVzZWQgZm9yIFRlc3RdW2lkNF0NCg0KIyMjIyBGaXR0aW5nIHRoZSBEZWVwIExlYXJuaW5nIE1vZGVsDQoNCkxhdW5jaCB0aGUgbWFjaGluZSBhZ2Fpbi4uLg0KDQpgYGB7ciwgZXZhbD1UUlVFLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFLCBpbmNsdWRlPVRSVUV9DQojIHRvIGxvYWQgdGhlIGxpYnJhcnkNCmxpYnJhcnkoaDJvKQ0KDQojIHRvIGluaXRpYWxpemUgdGhlICdtYWNoaW5lJw0KbG9jYWxIMk8gPSBoMm8uaW5pdCgpDQpgYGANCg0KIyMjIyBMb2FkIGRhdGFzZXRzIHRvIEgyTw0KDQpUaGVuIHdlIHdpbGwgZG93bmxvYWQgdGhlIGRhdGFzZXRzIGludG8gaDJvLiBSZW1lbWJlciBIMk8gaXMganVzdCBvcGVyYXRlZCBmcm9tIFIgYnV0IGl0J3MgYSBjb21wdXRlciBiZXNpZGVzIQ0KDQoNCmBgYHtyLCBldmFsPVRSVUUsIGluY2x1ZGU9VFJVRX0NCiMgSW1wb3J0IHRyYWluIGRhdGEgaW50byB0aGUgSDJPIGNsdXN0ZXINCnRyYWluX00xIDwtIGFzLmgybyh4ID0gREZfTTEsIGRlc3RpbmF0aW9uX2ZyYW1lID0gInRyYWluX00xIikNCg0KIyBBbHNvIGltcG9ydCBvdXIgdGVzdCBkYXRhc2V0cyBmb3IgTWFjaGluZXMgMiBhbmQgMy4uLg0KdGVzdF9NMiAgPC0gYXMuaDJvKHggPSBERl9NMiwgZGVzdGluYXRpb25fZnJhbWUgPSAidGVzdF9NMiIpDQp0ZXN0X00zICA8LSBhcy5oMm8oeCA9IERGX00zLCBkZXN0aW5hdGlvbl9mcmFtZSA9ICJ0ZXN0X00zIikNCg0KIyBNYWtlIHlvdXIgb3duIHJlc2VhcmNoIHVzaWduIERGX000IQ0KDQpgYGANCg0KDQojIyMjIEJ1aWxkaW5nIERlZXAgTGVhcm5pbmcgbW9kZWwgd2l0aCBhdXRvZW5jb2Rlcg0KDQpOb3csIG9uY2Ugd2Uga25vdyBob3cgb3VyIGRhdGEgbG9va3MgbGlrZSB3ZSBjYW4gc3RhcnQgdG8gZG8gb3VyIEFub21hbHkgTW9kZWwNCg0KYGBge3IsIGV2YWw9VFJVRSwgaW5jbHVkZT1UUlVFfQ0KDQojIFRyYWluIGRlZXAgYXV0b2VuY29kZXIgbGVhcm5pbmcgbW9kZWwgb24gIm5vcm1hbCIgDQojIHRyYWluaW5nIGRhdGEsIHkgaWdub3JlZCANCm5vcm1hbGl0eV9tb2RlbCA8LSBoMm8uZGVlcGxlYXJuaW5nKA0KIHggPSBuYW1lcyh0cmFpbl9NMSksIA0KIHRyYWluaW5nX2ZyYW1lID0gdHJhaW5fTTEsIA0KIGFjdGl2YXRpb24gPSAiVGFuaCIsIA0KIGF1dG9lbmNvZGVyID0gVFJVRSwgDQogaGlkZGVuID0gYyg1MCwyMCw1MCksIA0KIHNwYXJzZSA9IFRSVUUsDQogbDEgPSAxZS00LCANCiBlcG9jaHMgPSAxMDApDQpgYGANCg0KIyMjIyBDYWxjdWxhdGUgTVNFIGZyb20gdGhlIHRyYWluIGRhdGFzZXQNCg0KTGV0J3MgdXNlIHRoaXMgbW9kZWwgb24gb3VyIHRyYWluaW5nIGRhdGFzZXQuLi4NCg0KYGBge3IsIGV2YWw9VFJVRSwgaW5jbHVkZT1UUlVFfQ0KIyBjb21wdXRlciBlcnJvciBvZiB0aGUgbW9kZWwNCmgyby5hbm9tYWx5KG5vcm1hbGl0eV9tb2RlbCwgdHJhaW5fTTEpICU+JSBhcy5kYXRhLmZyYW1lKCkgJT4lIHBsb3QudHMoeWxpbSA9IGMoMCwgMikpDQpgYGANCg0KIyMjIyBHZXR0aW5nIHRvIGtub3cgd2hhdCBBSSBzZWU6DQoNCmBgYHtyLCBldmFsPVRSVUUsIGluY2x1ZGU9VFJVRX0NCiMgdmlzdWFsbHkgc2VlIGl0DQp0ZXN0X3JlY29uX00xIDwtIGgyby5wcmVkaWN0KG5vcm1hbGl0eV9tb2RlbCwgdHJhaW5fTTEpICU+JSBhcy5tYXRyaXgoKQ0KYGBgDQoNCmBgYHtyLCBldmFsPUZBTFNFLCBpbmNsdWRlPVRSVUV9DQpwbG90X2x5KHogPSB0ZXN0X3JlY29uX00xLCB0eXBlID0gInN1cmZhY2UiKQ0KYGBgDQoNCiFbUHJlZGljdGVkIG9uIFRyYWluIERhdGFzZXRdW2lkNV0NCg0KIyMjIyBEZXRlY3QgQW5vbWFseSB1c2luZyBNU0UgdmFsdWUNCg0KTm93IHdlIHdpbGwgdHJ5IHRvIHVzZSB0ZXN0IGRhdGFzZXRzIGZyb20gKiptYWNoaW5lIDIqKg0KDQpgYGB7ciwgZXZhbD1UUlVFLCBpbmNsdWRlPVRSVUV9DQojIENvbXB1dGUgcmVjb25zdHJ1Y3Rpb24gZXJyb3Igd2l0aCB0aGUgQW5vbWFseSANCiMgZGV0ZWN0aW9uIGFwcCAoTVNFIGJldHdlZW4gb3V0cHV0IGFuZCBpbnB1dCBsYXllcnMpDQpoMm8uYW5vbWFseShub3JtYWxpdHlfbW9kZWwsIHRlc3RfTTIpICU+JSAgYXMuZGF0YS5mcmFtZSgpICU+JSBwbG90LnRzKHlsaW0gPSBjKDAsIDEwKSwgdHlwZSA9ICJwIikNCmBgYA0KDQpBbmQgZm9yICoqbWFjaGluZSAzKioNCg0KYGBge3IsIGV2YWw9VFJVRSwgaW5jbHVkZT1UUlVFfQ0KaDJvLmFub21hbHkobm9ybWFsaXR5X21vZGVsLCB0ZXN0X00zKSAlPiUgIGFzLmRhdGEuZnJhbWUoKSAlPiUgcGxvdC50cyh5bGltID0gYygwLCAxMCksIHR5cGUgPSAicCIpDQpgYGANCg0KSXQgaXMgbm93IHRlbGxpbmcgdXMgdGhhdCBiZWhhdmlvdXIgb2YgTWFjaGluZSAyIGlzIG11Y2ggZGlmZmVyZW50IGZyb20gbWFjaGluZSAxLiBBbmQgb24gTWFjaGluZSAzIHdlIGhhdmUgYSBjbGVhcmx5IHBlYWtzIHRoYXQgYXJlIGRpc3Rpbmd1aXNoYWJsZSB0byByZWNvdmVyIGFub21hbHkhDQoNCioqTm90ZToqKiBZb3UgbWF5IHRyeSB0byBwcmFjdGljZSB3aXRoIE1hY2hpbmUgNCENCg0KDQojIyMjIFVzZSBBbm9tYWx5IG1vZGVsIHRvIHJlY29uc3RydWN0IFRlc3QgRGF0YXNldA0KDQpOb3cgd2UgY2FuIG9idGFpbiBwcmVkaWN0aW9ucywgb3IgcGh5c2ljYWwgdmFsdWVzIHVzaW5nIG91ciBtb2RlbC4gV2Ugc2hvdWxkIHByb3ZpZGUgdGhlIG1vZGVsIGFuZCB0ZXN0IGRhdGFzZXQNCg0KYGBge3IsIGV2YWw9VFJVRSwgaW5jbHVkZT1UUlVFfQ0KIyBOb3RlOiBUZXN0aW5nID0gUmVjb25zdHJ1Y3RpbmcgdGhlIHRlc3QgZGF0YXNldA0KdGVzdF9yZWNvbl9NMyA8LSBoMm8ucHJlZGljdChub3JtYWxpdHlfbW9kZWwsIHRlc3RfTTMpICU+JSBhcy5tYXRyaXgoKQ0KcGxvdF9seSh6ID0gdGVzdF9yZWNvbl9NMywgdHlwZSA9ICJzdXJmYWNlIikNCmBgYA0KDQp3ZSBjYW4gbm90IGRlZmluaXRlbHkgc2VlIHRob3NlIHBlYWtzIGFzIGNsZWFyIG5vdyBob3dldmVyIGxldCB1cyBiZSBzYXRpc2ZpZWQgd2l0aCB0aGUgbW9kZWwgcmVzdWx0IGFuZCB0aGluayBob3cgdG8gaW1wbGVtZW50IG91ciBEZWVwIExlYXJuaW5nIE1vZGVsIGluIFNoaW55QXBwDQoNCg0KIyMjIyBTYXZpbmcgRGVlcCBMZWFybmluZyBNb2RlbCBmb3IgdGhlIGZ1dHVyZSB1c2UNCg0KVG8gdXNlIG91ciBtb2RlbCBpbiBvdXIgU2hpbnlBcHAgd2Ugd2lsbCBzYXZlIGl0Li4uDQoNCmBgYHtyLCBldmFsPUZBTFNFLCBpbmNsdWRlPVRSVUV9DQoNCmlmKCFmaWxlLmV4aXN0cygid3d3L3RtcC9ub3JtYWxpdHlfbW9kZWwuYmluIikpew0KaDJvLnNhdmVNb2RlbChub3JtYWxpdHlfbW9kZWwsICJ3d3cvdG1wL25vcm1hbGl0eV9tb2RlbC5iaW4iKQ0KaDJvLmRvd25sb2FkX3Bvam8obm9ybWFsaXR5X21vZGVsLCAid3d3L3RtcCIsIGdldF9qYXIgPSBUUlVFKQ0KfQ0KYGBgDQoNCkFuZCBsZXQncyBub3QgZm9yZ2V0IHRvIHN3aXRjaCBvZmYgb3VyIGNsdXN0ZXIhDQpgYGB7ciwgZXZhbD1UUlVFLCBpbmNsdWRlPVRSVUV9DQpoMm8uc2h1dGRvd24ocHJvbXB0PSBGQUxTRSkNCg0KYGBgDQoNCiMjIyMgQ29uY2x1c2lvbg0KDQpJbiB0aGlzIGV4YW1wbGUgdGhlIEFub21hbHkgRGV0ZWN0aW9uIG1vZGVsIHdhcyBhYmxlIHRvIG91dHB1dCB0aGUgYW5vbWFseSBpbiByb3dzIDIxLTIzLiANCg0KSXQgbGVhcm5lZCBvbiB0aGUgcGF0dGVybiBvZiBtYW55IHZlY3RvcnMgYW5kIHdhcyBhYmxlIHRvIGRpc3Rpbmd1aXNoIHRoZSBhbm9tYWx5IGNvbWluZyBvbiBuZXcgZGF0YXNldA0KDQpQcmFjdGljYWwgdXNlIG9mIHRoaXMgbW9kZWwgY2FuIGJlIHRvIHVzIGZ1bmN0aW9uICoqaDJvLmFub21hbHkqKi4gSW4gY2FzZSB0aGUgTVNFIHZhbHVlIHdpbGwgYmUgaGlnaCAtIHRoZSBhbm9tYWx5IGlzIGRldGVjdGVkIQ0KDQojIyMjIE5leHQgc3RlcA0KDQpvdXIgbmV4dCBzdGVwIHdpbGwgYmUgdG8gcmVwZWF0IHRoZSBwcm9jZWR1cmUgYnV0IG9uIG91ciBtYWNoaW5lIGRhdGEuIA0KDQoqIHJlLWFycmFuZ2luZyBkYXRhIGFzIG1hdHJpeA0KKiBmaXR0aW5nIGRlZXAgbGVhcm5pbmcgbW9kZWxzDQoqIHRlc3RpbmcgdGhlIG1vZGVscw0KKiBzYXZpbmcgbW9kZWxzDQoqIGltcGxlbWVudGF0aW9uIGluIG91ciBTaGlueUFwcC4uLg0KDQojIyMjIHVzZWQgcmVmZXJlbmNlcw0KDQpleGFtcGxlIGZyb206IChodHRwczovL2R6b25lLmNvbS9hcnRpY2xlcy9hbm9tYWx5LWRldGVjdGlvbi13aXRoLWRlZXAtbGVhcm5pbmctaW4tci13aXRoLWgybylbaHR0cHM6Ly9kem9uZS5jb20vYXJ0aWNsZXMvYW5vbWFseS1kZXRlY3Rpb24td2l0aC1kZWVwLWxlYXJuaW5nLWluLXItd2l0aC1oMm9dDQoNCk1vcmUgcmVhZGluZzogKGh0dHBzOi8vZHpvbmUuY29tL2FydGljbGVzL3RoZS1iYXNpY3Mtb2YtZGVlcC1sZWFybmluZy1ob3ctdG8tYXBwbHktaXQtdG8tcHJlP2Zyb21yZWw9dHJ1ZSlbaHR0cHM6Ly9kem9uZS5jb20vYXJ0aWNsZXMvdGhlLWJhc2ljcy1vZi1kZWVwLWxlYXJuaW5nLWhvdy10by1hcHBseS1pdC10by1wcmU/ZnJvbXJlbD10cnVlXQ0KDQpBbmQ6IChodHRwczovL3NoaXJpbmcuZ2l0aHViLmlvL21hY2hpbmVfbGVhcm5pbmcvMjAxNy8wNS8wMS9mcmF1ZClbaHR0cHM6Ly9zaGlyaW5nLmdpdGh1Yi5pby9tYWNoaW5lX2xlYXJuaW5nLzIwMTcvMDUvMDEvZnJhdWRdDQoNCnBhcGVyOiAoaHR0cHM6Ly9hcnhpdi5vcmcvYWJzLzE3MDEuMDE4ODcpW2h0dHBzOi8vYXJ4aXYub3JnL2Ficy8xNzAxLjAxODg3XQ0KSW4gdGhpcyBsZWN0dXJlIHdlIHdvdWxkIGV4cGxvcmUgdGhlICd0ZWNobm9sb2d5JyBvbiB0aGUgc2FtcGxlIGFuZCB0cnkgdG8gZG8gdGhpcyBpbiAxMCBtaW4gbGVjdHVyZSENCg0KQXQgdGhlIGJvdHRvbSBvZiB0aGUgZG9jdW1lbnQ6DQoNCltpZDFdOiBwbG90cy9NMV90cmFpbi5wbmcgIkRhdGEgdXNlZCB0byBUcmFpbiB0aGUgbW9kZWwiDQpbaWQyXTogcGxvdHMvTTJfdGVzdC5wbmcgIkRhdGEgdXNlZCB0byBUZXN0IHRoZSBtb2RlbCAtIE1hY2hpbmUgMiINCltpZDNdOiBwbG90cy9NM190ZXN0LnBuZyAiRGF0YSB1c2VkIHRvIFRlc3QgdGhlIG1vZGVsIC0gTWFjaGluZSAzIg0KW2lkNF06IHBsb3RzL000X3Rlc3QucG5nICJEYXRhIHVzZWQgdG8gVGVzdCB0aGUgbW9kZWwgLSBNYWNoaW5lIDQiDQpbaWQ1XTogcGxvdHMvTTFfcHJlZC5wbmcgIldoYXQgQUkgc2VlIG9uIFRyYWluIERhdGEiDQoNCg0KDQo=